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Foreword 



id , 



This Technical Specification has been produced by the 3 Generation Partnership Project (3GPP). 

The contents of the present document are subject to continuing work within the TSG and may change following formal 
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an 
identifying change of release date and an increase in version number as follows: 

Version x.y.z 

where: 

x the first digit: 

1 presented to TSG for information; 

2 presented to TSG for approval; 

3 or greater indicates TSG approved document under change control. 

y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections, 
updates, etc. 

z the third digit is incremented when editorial only changes have been incorporated in the document. 



Introduction 



The present document is part of a TS-family covering the 3rd Generation Partnership Project; Technical Specification 
Group Services and System Aspects; Telecommunication management; as identified below: 

32.401 Performance Management (PM); Concept and requirements 

52.402 Performance Management (PM); Performance measurements - GSM 

32.404 Performance Management (PM); Performance measurements - Definitions and template 

32.405 Performance Management (PM); Performance measurements Universal Terrestrial Radio Access 
Network (UTRAN) 

32.406 Performance Management (PM); Performance measurements Core Network (CN) Packet Switched 
(PS) domain 

32.407 Performance Management (PM); Performance measurements Core Network (CN) Circuit 
Switched (CS) domain 

32.408 Performance Management (PM); Performance measurements Teleservice 

32.409 Performance Management (PM); Performance measurements IP Multimedia Subsystem (IMS) 

The present document is part of a set of specifications, which describe the requirements and information model 
necessary for the standardised Operation, Administration and Maintenance (OA&M) of a multi-vendor 3G-system. 

During the lifetime of a 3G network, its logical and physical configuration will undergo changes of varying degrees and 
frequencies in order to optimise the utilisation of the network resources. These changes will be executed through 
network configuration management activities and/or network engineering, see TS 32.600 [3], 

Many of the activities involved in the daily operation and future network planning of a 3G network require data on 
which to base decisions. This data refers to the load carried by the network and the grade of service offered. In order to 
produce this data performance measurements are executed in the NEs, which comprise the network. The data can then 
be transferred to an external system, e.g. an Operations System (OS) in TMN terminology, for further evaluation. The 
purpose of the present document is to describe the mechanisms involved in the collection of the data and the definition 
of the data itself. 
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Annex B has been added to help in the definition of new performance measurements that can be submitted to 3GPP for 
potential adoption and inclusion in the present document. Annex B discusses a top-down performance measurement 
definition methodology that focuses on how the end-user of performance measurements can use the measurements. 
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1 Scope 

The present document describes the measurements for UMTS and combined UMTS/GSM. 

TS 32.401 [1] describes Performance Management concepts and requirements. 

The present document is valid for all measurement types provided by an implementation of a UMTS network and 
combined UMTS/GSM network. 

Only measurement types that are specific to UMTS or combined UMTS/GSM networks are defined within the present 
documents. Vendor specific measurements types used in UMTS and combined UMTS/GSM networks are not covered. 
Instead, these could be applied according to manufacturer's documentation. 

Measurements related to "external" technologies (such as ATM or IP) as described by "external" standards bodies (e.g. 
ITU-T or IETF) shall only be referenced within this specification, wherever there is a need identified for the existence 
of such a reference. 

The definition of the standard measurements is intended to result in comparability of measurement data produced in a 
multi-vendor network, for those measurement types that can be standardised across all vendors' implementations. 

The structure of the present document is as follows: 

Header 1: Network Element (e.g. RNC related measurements); 

Header 2: Measurement function (e.g. soft handover measurements); 

Header 3: Measurements. 



References 



The following documents contain provisions which, through reference in this text, constitute provisions of the present 
document. 

• References are either specific (identified by date of publication and/or edition number or version number) or 
non-specific. 

• For a specific reference, subsequent revisions do not apply. 

• For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including 
a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same 
Release as the present document. 

[1] 3GPP TS 32.401: "Telecommunication management; Performance Management (PM); Concept 

and requirements". 

[2] 3GPP TS 52.402: "Telecommunication management; Performance Management (PM); 

Performance measurements - GSM". 

[3] 3GPP TS 32.600: "Telecommunication management; Configuration Management (CM); Concept 

and high-level requirements". 

[4] GSM 12.04: "Digital cellular telecommunications system (Phase 2+); Performance data 

measurements". 

[5] 3GPP TS 23.003: "Numbering, Addressing and Identification". 

[6] Victor R Basili and H. Dieter Rombach: "The TAME project: Towards improvement -oriented 

software environments", IEEE Transactions of Software Engineering, Vol. 14, No. 6, June 1988. 

[7] 3GPP TS 32.632: "Telecommunication management; Configuration Management (CM); Core 

Network Resources Integration Reference Point (IRP): Network Resource Model (NRM)". 



ETSI 



3GPP TS 32.404 version 7.4.0 Release 7 7 ETSI TS 1 32 404 V7.4.0 (2007-1 0) 

[8] 3GPP TS 32.642: "Telecommunication management; Configuration Management (CM); UTRAN 

network resources Integration Reference Point (IRP): Network Resource Model (NRM)". 

[9] 3GPP TS 32.622: "Telecommunication management; Configuration Management (CM); Generic 

network resources Integration Reference Point (IRP): Network Resource Model (NRM)". 



Definitions and template 



3.1 Definitions 

For the purposes of the present document, the following terms and definitions apply: 

"(n-1) out of n" approach: 

The measurements result values generated by a NE can be obtained in a number of different ways. Therefore, the 
"(n-1) out of n approach" has been defined in order to avoid redundancy in the measurements. 



The "(n-1) out of n approach" allows a vendor to choose any (n-1) out of the n defined counters for 
implementation but some cho: 
calculated in post-processing. 



implementation but some choices can offer more detailed information than others. The missing n th value can be 



If multiple measurements are included in one template, then the applicability of the "(n-1) out of n" scenario are 
mentioned in template item A with the following sentence "The n measurement types defined in item E are 
subject to the "(n-1) out of n approach" ". The item D will specify the measurement result per measurement type 
specified in template item E. 

If the measurements that are applicable to the "(n-1) out of n" scenario are defined in separate templates, then 
they will be grouped together into a common clause of the TS, and the applicability of the approach will be 
mentioned in the clause that groups the measurements. 

Examples of measurements which are subject to the "(n-1) out of n" approach are provided in the annex A. 

Measurement community 

Several measurement communities are defined in the present document to identify the end users of system 
measurements. Each measurement should be defined to address the needs of at least one of these user communities. 

Six communities have been identified so far: 

Network Operator's Business Community 
Network Operator's Maintenance Community 
Network Operator's Traffic Engineering Community 
Network Operator's Customer Care Community 
Equipment Vendor's Performance Modelling Community 
Equipment Vendor's Development Engineering Community 

A comprehensive description of measurement communities is provided in Annex B. The user communities names are a 
composite of the various terms used in the industry and might be subject to modification or refinement in future 
releases. 

Measurement family 

The measurement names defined in the present document are all beginning with a prefix containing the measurement 
family name (e.g. RAB.AttEstabCS.Conv, MM.AttGprsAttach). This family name identifies all measurements which 
relate to a given functionality and it may be used for measurement administration (see TS 32.401 [1]). 

The list of families currently used in the present document is as follows: 

HHO (measurements related to Hard Handover) 

MM (measurements related to Mobility Management) 

RAB (measurements related to Radio Access Bearer management) 

RRC (measurements related to Radio Resource Control) 
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3.2 



SM (measurements related to Session Management) 
SMS (measurements related to Short Message Service) 

Abbreviations 



For the purposes of the present document, the following abbreviations apply: 

3G 3 rd Generation 

3GPP 3G Partnership Project 

ASN. 1 Abstract Syntax Notation 1 

ATM Asynchronous Transfer Mode 

BER Basic Encoding Rules 

CN Core Network 

DTD Document Type Definition 

EGQM Enhanced Goal, Question, Metric 

EM (Network) Element Manager 

GQM Goal, Question, Metric 

IEEE Institute of Electrical and Electronics Engineers, Inc. 

Itf Interface 

ITU-T International Telecommunication Union - Telecommunications Standardisation Sector 

NE Network Element 

NM Network Manager 

OA&M Operation, Administration and Maintenance 

OS Operations System (EM, NM) 

OSI Open Systems Interconnection 

PM Performance Management 

QoS Quality of Service 

RNC Radio Network Controller 

UMTS Universal Mobile Telecommunications System 

UTRAN Universal Terrestrial Radio Access Network 

You can find below a list of abbreviations used within the measurement types for field E of the measurement template 
(see subclause 3.3). 



Att 


Attempt(s,ed) 


Conn 


Connection 


CS 


Circuit switched 


Conv 


Conversational 


Estab 


Establish (ed,ment) 


FDD 


Frequency Division Duplex 


HHO 


Hard Handover 


HO 


Handover 


Inter 


Inter 


Intra 


Intra 


Max 


Maximum 


MM 


Mobility Management 


Out 


Outgoing 


PS 


Packet switched 


RAB 


Radio Access Bearer 


RNC 


RNC 


RRC 


Radio Resource Control 


SGSN 


SGSN 


Sub 


Subscriber 


Succ 


Success(es,ful) 


UTRAN 


UTRAN 
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3.3 Measurement definition template 

Following is the template used to describe the measurements contained in this subclause. 
C.x.y. Measurement Name (clause header) 

This is a descriptive name of the measurement type that is specified as clause C.x.y of the present document. 

The measurement name shall be written in lower-case characters except abbreviations (e.g. RNC). 

A measurement name can apply to one or more measurements. If the measurement name applies to several 
measurements then all fields of the template will take this into account. 

Measurement names shall not exceed 64 characters in length and should be constrained to 32 characters 
maximum. Exceptions greater than 32 characters are allowed but should be kept to a minimum and only made 
where necessary. 

a) Description 

This subclause contains an explanation of the measurement operation. 

b) Collection Method 

This n contains the form in which this measurement data is obtained: 

CC (Cumulative Counter); 

GAUGE (dynamic variable), used when data being measured can vary up or down during the period of 
measurement; 

DER (Discrete Event Registration), when data related to a particular event are captured every n* event is 
registered, where n can be 1 or larger; 

- SI (Status Inspection). 

c) Condition 

This subclause contains the condition which causes the measurement result data to be updated; 
This will be defined by identifying protocol related trigger events for starting and stopping measurement 
processes, or updating the current measurement result value. Where it is not possible to give a precise condition, 
then the conditional circumstances leading to the update are stated. 

If a measurement is not available for FDD or TDD, then the measurement description shall contain a statement. 

If a measurement is related to 'external' technologies, this subclause shall give a brief reference to other standard 
bodies. 

d) Measurement Result (measured value(s), Units) 

This subclause contains a description of expected result value(s) (e.g. a single integer value). If a measurement is 
related to 'external' technologies, this subclause shall also give a brief reference to other standard bodies. 

The definition applies for each measurement result. 

e) Measurement Type 

This subclause contains a short form of the measurement name specified in the header, which is used to identify 
the measurement type in the result files. 

The measurement names are dotted sequences of items. The sequence of elements identifying a measurement is 
organised from the general to the particular. 

The first item identifies the measurement family (e.g. HHO, RAB, SMS). Note that this family may also be 
used for measurement administration purpose. 

The second item identifies the name of the measurement itself. 
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Depending on the measurement type, additional items may be present to specify subcounters (failure causes, 
traffic classes, min, max, avg, G, U ...)• In case of multiple additional items, they are also represented as a 
dotted sequence of items. When available, the template will describe to which standard it is referring to for 
these additional items (e.g. cause, traffic class). Otherwise, the additional item semantics must be described 
in details in the present document. 

Standardised causes will be a number. This number shall be derived according to which of the following 
rules applies: 

For the standardised causes with numeric values explicitly specified in the refenece spefication, the subcounter 
name will be the number assigned to this cause in this reference specification. 

For the standardised causes without numeric values explicitly specified in the reference spefication, but where 
the causes are identified, the subcounter name shall be an number from 1 to n mapped in an incremental 
sequence to each of the specified causes following top down sequence in the order they are identified in the 
reference specification (e.g. RRC.ConnEstab.l, 1 idenfies the establishment cause 'Originating Conversational 

Call', see TS 25.331); 

For the standardised causes without numeric values explicitly specified in the reference spefication and the 
causes identified and the causes have been divided into different cause groups, the subcounter should be defined 
as the form of "Cause group". Cause, where: 

o the subcounter name of "Cause group" shall be an incremental number from 1 to n to identify each 
cause group specified in a top down sequence following the order they are identified in the reference 
specification, 

o the subcounter name of cause within this cause group shall be an incremental number from 1 to n to 
identify each cause within the group specified in a top down sequence following the order they are 
identified within the cause group in the reference specification (e.g. RLM.FailRLSetupIub.2.1, 2.1 
indenfies the cause Transport resource unavailable' in the cause group Transport Layer Cause', see TS 
25.433). Subcounter "Cause group". sum is permitted to indentify the aggregate of measurement values 
of all the causes within the cause group (e.g. RLM.FailRLSetupIub. 1 .sum, 1 .sum identifies the 
aggregate of all the causes within the cause group 'Radio Network Layer Cause', see TS25.433). 

The non standardised causes should be a string (e.g. RRC.ConnEstab.NoReply). 

The set of values issued for a measurement does not depend on the associated collection method (CC, SI, Gauge, 
DER). For instance, a gauge collected counter does not necessarily provide min, max, average values. 

The vendor-specific UMTS and combined GSM/UMTS measurement names will all begin with the VS prefix. 

In addition, it is recommended that a prefix is added for non-UMTS measurements: 

Q3 for Q3 measurements; 

- MIB for IETF measurements (ATM, IP); 
OS for other standards measurements. 

The 3GPP standardised measurements name must not commence with the above prefixes. 
Examples of valid measurement names are: 

- VS.HO.InterSGSNReject.NoResource; 

- ATM. ATML.IngressCells; 

- HHO.SuccOutlntraCell; 

- MMAttachedSubs.Max; 

- RAB.EstabAttCS.Conversational; 

RRC. ConnEstab.Cat«e 

where Cause identifies the failure cause. 
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Abbreviations to be used within measurement types can be found in subclause 3.2 of the present document. 

f) Measurement Object Class 

This subclause describes the measured object class (e.g. UtranCell, RncFunction, SgsnFunction). The object 
class used for this purpose shall be in accordance with the Network Resource Model defined in 
3GPP TS 32.622 [9], TS 32.632 [7] and TS 32.642 [8]. 

For object classes currently not defined in CM, the present document defines its own nomenclature (e.g. RA, 
LAC). 

It is possible to use the same measurement name for a standardized measurement type implemented at a different 
object class level than the one defined in the Standard. The same measurement type can apply to one or more 
measurements for which all fields of the measurement template are the same except the clause f) "Measurement 
Object Class". For instance, a measurement which uses the same template as a given measurement type but 
relates to another or different object classes (e.g. UtranCell instead of UtranRelation, or RncFunction and 
UtranCell) shall have the same name. 

g) Switching Technology 

This subclause contains the Switching domain(s) this measurement is applicable to i.e. Circuit Switched and/or 
Packet Switched. 

h) Generation 

The generation determines if it concerns a GSM, UMTS, combined (GSM+UMTS) or IMS measurement. 

- GSM: pure GSM measurement; it only counts GSM events. In a combined (GSM+UMTS) NE the count 
would be exactly the same as in a pure GSM NE. In a pure UMTS NE this counter does not exist; 

- UMTS: pure UMTS measurement; it only counts UMTS events. In a combined (GSM+UMTS) NE the count 
would be exactly the same as in a pure UMTS NE. In a pure GSM NE this counter does not exist; 

- GSM/UMTS: measurement applicable to both GSM and UMTS systems; in a combined (GSM+UMTS) NE 
separate subcounts for GSM and/or UMTS events can be obtained; 

Combined: measurement applicable to combined GSM and UMTS systems, but regardless of whether the 
measured event occurred on the GSM or UMTS part of the system. This means that in a combined NE only 
one total (i.e. GSM+UMTS) count is obtained for the measured event. 

The above aspects are also reflected in the measurement type name in template item E by adding a "G" to the 
GSM measurements and "U" to the UMTS measurements. 

IMS: measurement applicable to IMS. 

NOTE: The 2G component of combined 2G/3G equipment may actually choose to implement GSM 

measurements according to the present document or GSM12.04 [4] / TS 52.402 [2], based on GSM 
specifications. 

i) Purpose 

This optional clause aims at describing who will be using the measurement. It is proposed to indicate in this 
clause the targeted categories of users based on the measurement user communities described in Annex B. 

When available, this clause provides additional information on the interest of the measurement but is however 
purely indicative. 



ETSI 



3GPP TS 32.404 version 7.4.0 Release 7 12 ETSI TS 132 404 V7.4.0 (2007-10) 

3.4 Definition of private Object Classes 

Private Object Classes are Object Classes which are needed for PM purposes, but that are not yet defined by CM. 

3.4.1 RA 

The Object Class RA (Routing Area) is needed to conduct measurements on RA level. For the purpose of the present 
document, the Routing Area should be encoded in the file format as the concatenation of the MCC, MNC, LAC and the 
RAC, in decimal notation. See further definition of Routing Area Identification in TS 23.003 [21]. Since LAC is a 2 
byte number (00000-65535), 5 characters are needed in the moid PrintableString. Since RAC is a 1 byte number 
(000-255) 3 characters are needed in the moid PrintableString. MCC is 3 digits and MNC 2 or 3 digits. The 
concatenated moid PrintableString will always contain 14 characters. In the case where MNC has the length 2, a leading 
underscore character will be added. 

a) EXAMPLE 1 : LAC = Hexadecimal 4E20 = Decimal 20000; 

RAC = Hexadecimal BE = Decimal 190; MCC = Decimal 046; MNC = Decimal 01 
moid = "046_0 120000 190". 

The Object Class RA (Routing Area) is needed to conduct measurements on RA level. For the purpose of the present 
document the Routing Area should be encoded in the file format as the concatenation of the LAC and the RAC, in 
decimal notation. Since LAC is a 2 byte number (00000-65535) 5 characters are needed in the moid PrintableString. 
Since RAC is a 1 byte number (000-255) 3 characters are needed in the moid PrintableString. Hence concatenated moid 
PrintableString will always contain 8 characters. 

EXAMPLE 2: LAC = Hexadecimal 4E20 = Decimal 20000; 
RAC = Hexadecimal BE = Decimal 190; 
moid = "20000190". 

3.4.2 MMS Relay/Server 

The Object Class MMS Relay/Server is needed to conduct measurements on MMS Relay/Server level. No specific 
moid structure is defined for the MMS Relay/Server. The moid is a PrintableString which contains a maximum of 
20 characters. 

3.5 Management of per cause measurements 

Per cause measurements may lead in certain cases to a lot of subcounters which will increase substantially the size of 
the measurement report file. Since all per cause measurements are not necessarily useful to the end-user, two options 
are possible for the management of the corresponding subcounters: 

support all the subcounters corresponding to the cause codes as defined in the 3GPP standards. In that case, the 
sum of all supported per cause measurements is equal to the total sum across all subcounters; 

support only a subset of the subcounters (allowed only if the cause codes are specified in 3GPP standards). In 
that case, the first value of the result sequence must be the total sum across all the cause codes as defined in 
3GPP standards. This implies that all subcounters of a given measurement type appear as uninterrupted sequence 
in the result file. The keyword .sum placed behind the measurement type is used to identify the sum subcounter. 
The choice of the supported cause codes is manufacturer dependent. 
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Annex A (informative): 

Examples for "(n-1) out of n" approach 

The measurements result values generated by a NE are often redundant, or the info contained in the measurement 
results can be obtained in a number of different ways. 

The "(n-1) out of n" approach allows a vendor to implement a subset of 3GPP defined measurements, for example if 
there exists a relation (A+B=C) then any 2 out of 3 defined measurements A, B, C are sufficient information to 
calculate the third (n=3). In case there exists a relation (A+B+C=D), then any 3 out of the 4 would suffice, and the same 
kind of approach would be applicable. 

A.1 Attempt/success/failure procedure measurements 

Consider the number of attempts to start a specific procedure (e.g. RRC connection establishment). Some of these 
attempts will fail, some will be successful. Three different counters can be defined to measure these procedures: an 
attempt counter, a success counter, and a failure counter, but in fact only 2 may be provided, since we have the fixed 
relation (#success + #failure) = #attempt. 

It is to be noted that all combinations do not provide the same level of details. For example, in the case only #attempt 
and #success are provided, it will not be possible to retrieve the detailed failure causes. 

The three measurement types defined in clause 4.4 of TS 32.405 are subject to the "(n-1) out of n" approach with n=3: 

Attempted RRC connection establishments. 
Failed RRC connection establishments. 
Successful RRC connection establishments. 

The "(n-1) out of n" approach is also applicable for more complex measurements split according to a specific criterion, 
e.g. Queuing. For example, the CS measurements described in clause 4.1 of TS 32.405 are subject to a 4 out of 5 
approach: 

Attempted RAB establishments for CS domain. 
Successful RAB establishments without queuing for CS domain. 
Failed RAB establishments without queuing for CS domain. 
Successful RAB establishments with queuing for CS domain. 
Failed RAB establishments with queuing for CS domain. 

Any of the five measurements can be calculated from the four others but all combinations will not provide the same 
level of details (e.g. failure causes). 

A.2 GSM/UMTS combined measurements 

With relation to the field H of the measurement template, a measurement indicated with GSM/UMTS is an example of 
the "(n-1) out of n" approach with n=3 since (GSM + UMTS) = Combined. 

In that case, all concerned measurements are included in the same template but the vendor may provide only 2 
subcounters out of 3. 

The measurement described in subclause 4.6.1 of TS 32.406 is subject to the "(n-1) out of n" approach with n=3: 

SM.AttActPdpContext (attempted context activation procedures with no distinction between GSM and UMTS). 
SM.AttActPdpContext.G (attempted context activation procedures for GSM only). 
SM.AttActPdpContext. U (attempted context activation procedures for UMTS only). 
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A.3 Embedded "(n-1) out of n" approaches 

It is also possible to combine the approaches described above. For example, the measurements described in 
subclause 4.5 of TS 32.406 are subject to the "(n-1) out of n" approach at two levels. 

Firstly, measurements are split according to the CS/PS domain, for example: 

attempted CS SMS mobile originating; 
attempted PS SMS mobile originating; 
attempted SMS mobile originating; 

where any of the three measurements can be calculated from the two others. 

Secondly, each measurement provides 3 subcounters, for example for Attempted CS SMS mobile originating 

SMS.AttMoCS; 

SMS.AtfMoCS.G; 

SMS.AttMoCS.U; 

where any of the three subcounters can be calculated from the two others. 
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Annex B (informative): 

Top-Down Performance Measurement Definition Process 

B.1 Scope of this annex 

Performance measurements within wireless telecommunications networks are required in order to meet the needs of the 
diverse community of end-users of those measurements. New features develop, networks evolve and operating 
conditions change without sufficient consideration given to the measurements needed to keep the network running 
efficiently. While Equipment Vendors define measurements to satisfy their particular needs, other perspectives, 
especially the voice of the Network Operator, are sometimes lost during Equipment Vendor development processes. 
Similarly, Network Operators sometimes request measurements without fully understanding who will be using the data 
or what actions those people will take based on the collected data. A coherent, simple, top-down methodology for 
defining performance measurements is lacking in the telecommunications industry. 

This annex proposes a methodology to handle the above problems. In particular, multiple user communities have been 
defined representing the end-users of system measurements. Performance goals and measurements are defined 
considering these same user communities. The definition includes identification of specific problem scenarios and 
corrective actions to be taken by the appropriate user community. 

Measurements defined using this methodology can be contributed to 3GPP SA5 for potential adoption and inclusion in 
the present document. It is believed that this methodology will help reduce development costs for the Equipment 
Vendors and reduce operational costs for the Network Operators. 

B.2 Overview 

Performance measurements are important to the proper and efficient functioning of wireless telecommunications 
networks. They have numerous uses related to resource utilization, expansion planning, network optimisation, operating 
problem diagnosis and network availability monitoring. For the wireless telecommunications world, product 
performance measurements are necessary to support multiple communities of users. 

In addition, once performance measurements are defined for a wireless telecommunication network they must be 
maintained. The evolution of a wireless telecommunication network for capacity increases and feature extensions leads 
to the evolution of the collected measurements. Performance measurements need to be added, modified and made 
obsolete from the overall measurement repository. These changes must be defined completely and accurately to meet 
the requirements of each community of users. 

The development of a performance measurement life cycle process to oversee this need is proposed in this annex. The 
life cycle process addresses the multiple user communities whose perspectives are needed to supply the requirements 
for the performance measurements. 

The proposed performance measurement life cycle process is a usage-based process. The basic Goal, Question, Metric 
(GQM) method is enhanced to define problem scenarios and corrective actions. These descriptions are not only used to 
filter out proposals for performance measurements that provide no defined benefit, but also support user community 
training in the use of the performance measurements. 

The remainder of this annex is organised as follows. 

Clause B.3 defines Measurement User Communities for wireless telecommunications. 
Clause B.4 discusses the GQM and the Enhanced GQM methods. 
Clause B.5 discusses the measurement life cycle process. 
Clause B.6 provides conclusions. 
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B.3 Measurement User Communities 

One objective of Performance Management as a functional subset of operations and maintenance processes is to define 
sets of measurements. Typical definition criteria revolve around measuring activity within the network in terms of 
volume, speed and accuracy. While this approach produces measurement data it does not completely address the needs 
and uses of the multiple consumers of network performance measurement information. The Enhanced GQM 
methodology extends the measurement definition criteria to better satisfy multiple groups with diverse needs for these 
measurements. 

A qualitative judgement as to the efficacy of a Performance Management subsystem is how well served these different 
groups are by the measurements provided. To better understand these needs, five generic categories of users, outlined 
definitions and examples of their needs and requirements for measurements taken from their wireless 
telecommunications network are defined. These groups are referred to as measurement user communities. These six 
communities are: 

1 . Network Operator's Business Community 

2. Network Operator's Maintenance Community 

3. Network Operator's Traffic Engineering Community 

4. Network Operator's Customer Care Community 

5. Equipment Vendor's Performance Modelling Community 

6. Equipment Vendor's Development Engineering Community 

B.3.1 Network Operator Business Community 

The first measurement user community is the Network Operator's Business Community. This community is defined 
under the assumption that the wireless telecommunications network is fully operational, adequately engineered for 
traffic load per quality of service definitions and in commercial service. The primary objective of this community is to 
guarantee the financial health and welfare of the Operating Company. They expect a properly configured wireless 
telecommunications network to supply the revenue per subscriber unit necessary to meet their financial goals. 

An understanding of the elasticity of demand can help the Business Community maximize profits within their product 
pricing strategy as they alter prices according to various mixes of services. Typical measurements of interest to this 
community are those based on the actual volumes of calls completed by service type. This call volume information can 
lead to trends of usage over time. Correlation between price mix and call volumes can help to identify pricing strategies 
geared towards increasing revenue per subscriber unit. 
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B.3.2 Network Operator Maintenance Community 

The second measurement user community is the Network Operator's Maintenance Community. This community is 
defined under the assumption that the wireless telecommunications network is less than fully operational, adequately 
engineered for traffic load per quality of service definitions and in commercial service. The primary objective of this 
community is to reduce Mean Time to Repair faults that occur within the network equipment of the Operating 
Company. 

The baseline metric for this community is the availability of the network equipment, where availability is composed of 
the sum of scheduled and unscheduled outages to the network equipment. Unscheduled outages are influenced by the 
inherent hardware and software quality of the products provided to the operating company. While the Maintenance 
Community has no direct control over that quality, they do have control over the second component of scheduled 
outage, Mean Time to Repair. 

Mean Time to Repair is influenced by the Mean Time to Detect a fault. This community of user's defines measurements 
that support detecting or predicting faults within the network equipment. 

Measurements that support this community can come from places other than the network equipment, itself. Several 
Operating Companies have been observed building information systems based on the data provided by Call Detail 
Records and Billing Records. Correlation is sought within these data between call faults and location within the 
Network. Detection of these faults serves a dual purpose: it allows the Operating Company a view of performance at the 
level of their Network Operator (the subscriber) and it allows the Maintenance Community to target specific network 
equipment for repair. 

B.3.3 Network Operator Traffic Engineering Community 

The third measurement user community is the Network Operator's Traffic Engineering Community. This community is 
defined under the assumption that the wireless telecommunications network is fully operational, inadequately 
engineered for current or future traffic load per quality of service definitions and in commercial service. The primary 
objective of this community is to keep the capacity of the network equipment within 1) the Operating Company's design 
criteria for the quality of service based on growth of the subscriber base, 2) changes in usage patterns based on pricing 
strategies and 3) introduction of new services. 

The baseline metric for this community is the trend in utilization of the network equipment. A fully instrumented 
network would allow the Operating Company to understand the trend in performance of their principle capital 
investment and any leased services. As these trends pass thresholds of performance, purchasing decisions or volume 
pricing discounts could be triggered. 

B.3.4 Network Operator Customer Care Community 

The fourth measurement user community is the Network Operator's Customer Care Community. This community is 
defined under the assumption that the wireless telecommunications network is fully operational, functioning at a less 
than optimal level resulting in end-user dissatisfaction and in commercial service. The primary objective of this 
community is interfacing with the end-user customers of offered services for the purpose of establishing and 
maintaining end-user customer satisfaction. This may include customer care responsibilities such as Customer 
Relationship Management (CRM), Service Level Agreement (SLA) management, quality of service (or QoS) 
management, etc. 

This community is interested in defining measurements related to the end-user customer experience with the network 
Operator's offered services in the areas of CRM, SLA, QoS, problem reports, etc. Decisions on how to best handle 
customer dissatisfaction or how to keep customers from becoming dissatisfied are based on these types of 
measurements. 
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B.3.5 Equipment Vendor Performance Modelling Community 

The fifth measurement user community is Equipment Vendor's Performance Modelling Community. This community is 
defined under the assumption that the wireless telecommunications network is fully operational, adequately engineered 
for traffic load per quality of service definitions and in some level of call capable service. The primary objective of this 
community is to guarantee that the models used during analysis and design phases conform to real-world observations 
of the network equipment of the Operating Company. 

While this community is not within the Operating Company it still provides beneficial service to the Operating 
Company by managing the development of subsequent features that are in line with the actual performance 
characteristics of the network. Many decisions within the development life cycle depend on models developed prior to 
shipping the product. These models need to be calibrated to network performance once the product is released. 
Definition of measurements in concert with calibrating these models requires the direct involvement of the people 
developing the models. 

The network that transports Network Management data often is the same network that carries call control traffic. 
Clearly, the knowledge of volume levels of this traffic during anomalous operating conditions is important in order to 
understand the total impact to call processing. This community would define measurements to allow the monitoring of 
this type of phenomena. 
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B.3.6 Equipment Vendor Development Engineering Community 

The sixth measurement user community is Equipment Vendor's Development Engineering Community. This 
community is also defined under the assumption that the wireless telecommunications network is fully operational, 
adequately engineered for traffic load per Quality of Service or Service Level Agreement definitions and in commercial 
service. The primary objective of this community is to guarantee that the implementations of the designs conform to 
real-world observations of the network equipment of the Operating Company. 

While this community also is not within the Operating Company, it still provides beneficial service to the Operating 
Company. The implementation of new algorithms carries some finite risk of performance in the Network Operator 
environment versus the lab environment. Many times simulators of network activity are developed to support the 
verification of these algorithms. These simulations need to be calibrated to network performance once the product is 
released. Definition of measurements in concert with calibrating these simulations requires the direct involvement of the 
people developing the simulations. 



B.3.7 User Community Conclusion 



The six measurement communities are broken into four Network Operator based groups and two Equipment Vendor 
groups. However, experience shows that the measurements defined for these groups are not mutually exclusive. Other 
groups may also use measurements needed by a particular group for the same or different purposes. Thus, the accurate 
definition of the measurements and how to use them is necessary to allow the Network Operator to properly combine 
these measurements into more complex analyses. 
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B.4 Enhanced GQM 



The Goal, Question, Metric (GQM) method requires measurement user communities to help define goals and metrics. 
This clause first presents the standard GQM method and then presents an enhanced GQM method, which provides the 
measurement user communities a broader understanding of how metrics are used. The term 'Metric' in GQM means the 
same as 'measurement'. 



B.4.1 GQM Methodology 



Basili and Weiss [23] and others originally proposed the GQM method. This methodology provides a systematic 
approach for defining metrics that can be collected and analysed to determine whether or not a goal has been reached. 
This methodology was originally created for quality assurance of software development processes, but has been applied 
to other areas. GQM is comprised of the following three steps. 

1 . Identify and define Goals for a particular group; 

2. Refine goals into quantifiable Questions; 

3. Define Metrics that will answer the questions. 

Goals are defined in terms of a purpose and a perspective. The purpose specifies the object to be analysed and why it 
will be analysed. The perspective specifies the relevant aspects of the object and which measurement user community is 
interested in the aspects. 

Execution of the GQM methodology results in the formation of a GQM model. A GQM model contains the set of 
defined Goals, Questions and Metrics. A GQM model provides trace-ability from the goals to the associated metrics. 
Figure B.4. 1.1 shows an example of a GQM model. 
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Goal A 



Goal B 
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Figure B.4.1.1: GQM Model 

GQM definition templates are often used to help produce consistent goal, question and/or metric definitions. An 
example of a Goal template is shown below [6]: 

Purpose: To (characterize evaluate, predict, motivate, etc.) the (process product model, metric, etc.) in order to 
(understand, assess, manage, engineer, learn, improve, etc.) it. 
Example: To evaluate the system testing methodology in order to improve it. 

Perspective: Examine the (cost, effectiveness, correctness, defects, changes, product metrics, reliability, etc.) from 
the point of view of the (developer, manager, Network Operator, corporate perspective, etc.). 
Example: Examine the effectiveness from the developer's point of view. 
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B.4.2 Enhanced GQM (EGQM) Methodology 

As it stands, the GQM methodology works well for defining metrics, but falls short in several areas. The original GQM 
methodology was enhanced to better fit within the wireless telecommunications industry for the following reasons: 

a) Allow wireless measurement user communities to specify their needs at the beginning of the performance 
measurement life cycle rather than waiting for product to be delivered. 

b) Allow wireless measurement user communities to understand what performance measurements are being 
designed for them in time to modify the associated collection, analysis and reporting processes. 

c) Allow wireless measurement user communities to understand how they should analyse collected measurement 
data and what actions they should take when analysis has been completed. 

d) Provide criteria for rejecting unnecessary goals, useless measurements, or measurements that can not be properly 
collected, analysed or understood. 

e) Provide criteria for architecting metrics into the appropriate wireless network device (based on network traffic 
capacity, device CPU and memory capacity, data collection capabilities, etc.). 

f) Allow for consistent measurement definition by providing Enhanced GQM model definition and measurement 
definition templates. 

g) Help reduce development costs for Equipment Providers and reduce operational costs for the Network Operator. 
The Enhanced GQM, or EGQM, methodology is comprised of the following four steps. 

1 . Identify and define measurement goals for a particular measurement user community; 

2. Refine measurement goals into quantifiable problem scenarios; 

3. Define measurements that will determine if the goal is being accomplished; 

4. Define corrective actions. 

EGQM's first and third steps are similar to GQM's first and third steps. EGQM's second step is different than GQM's 
second step in that it focuses on problem scenarios associated with the goal rather than on questions associated with the 
goal. Problem scenarios are descriptions of real world problems the measurement user community has or will 
experience. Each problem scenario represents a particular aspect of the associated goal. Problem scenarios include 
definitions of any formulas that will allow the measurement user community to analyse the problem scenario after 
metric data has been collected from the field. EGQM's fourth step is new. Corrective actions are descriptions of what 
the measurement user community should do based on analysis of metric data collected from the resulting wireless 
network. 

Execution of the EGQM methodology results in the formation of an EGQM model. An EGQM model contains the set 
of defined goal, problem scenarios, metrics and corrective actions. An EGQM model also provides trace-ability from 
the goals to the associated corrective actions. Figure B.4.2. 1 shows an example of an EGQM model. 
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Figure B.4.2.1: EGQM Model 

EGQM has definition templates for producing an EGQM model and for defining metrics. The EGQM model definition 
template is shown in Table B 1 . The EGQM metric definition template that is useful for 3GPP S A5 activities is defined 
in subclause 3.3 of the present document. 



Goal 



Problem Scenario(s) 

Required Metric(s) 

Corrective Action(s) 



Table B.4.2.1 : EGQM Model Definition Template 

Provides the name of goal and non-ambiguous definition of what needs to be accomplished. 

Also provides the measurement user communities the goal is associated with. 

Provides a description of the problem scenario associated with the goal. Contains a description 

of how performance measurements will be used by the user in order to meet the goal. 

Provides a list of metrics required to assess the problem scenario to see if the goal is being 

accomplished. 

Provides descriptions of actions the user can execute based on data collected from the 

wireless network. Contains descriptions of expected metric data values and how those values 

work with the Problem Scenarios definitions. 



As described in clause B.3, six measurement user communities have been defined for the wireless telecommunications 
industries. EGQM supports all six communities. Representatives from each community participate in all four steps of 
the EGQM methodology. This allows user communities to specify exactly what they need and/or want and to know 
exactly how they will use the metrics before any software is developed. Participation in the EGQM process increases 
Network Operator satisfaction through early definition of operational practices (including corrective actions) and 
increases product knowledge within the Network Operator organization. 

The EGQM model definition and metric definition templates provide the mechanism to reject unnecessary goals, 
useless metrics, or metrics that can't be properly collected or computed. Reasons for the rejection of a goal include the 
following: 

a) Non-ambiguous goal definition could not be determined; 

b) Problem scenarios could not be determined; 

c) Definition of how performance measurement will be used within a problem scenario could not be determined; 

d) Corrective actions could not be determined; 

e) Metrics could not be defined to support problem scenario definitions; 

f) Required metrics could not be architected into network devices for any of the following reasons: 

1) Network device could not collect metric due to CPU utilization issues; 

2) Network device could not collect and/or store metric due to memory issues; 

3) Network could not support the uploading of measurement data from network devices to network manager; 

4) Network manager could not collect and/or store measurement data due to memory issues. 
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B.5 Measurements Life Cycle Process 

If the uses of performance measurements were confined to feature releases and occasional changes to those features, 
then EGQM would suffice. However, user community needs evolve, operating conditions change, performance models 
are validated, new services are introduced, etc. As these conditions change, performance measurements may need to 
change. Such considerations point to the need for a complete measurements life cycle model. 

A simple life cycle model to handle performance measurement changes is depicted in Figure B.5.1. New performance 
measurement goal and metric definitions are provided through new features. These are made available with major 
releases. 



New 




Functional 



o 



Obsolete 



Modify 



V 



Deleted 



Figure B.5.1 : Measurement Life Cycle 

Performance measurements may need to be periodically reviewed. Goal and metric definition updates made during this 
process are generally instantiated at major releases. When metrics are no longer useful they can be made obsolete and 
eventually deleted. A waiting period between obsolescence and deletion allows user communities time to implement 
and test out new metrics and analyses that are meant to replace existing metrics and analyses. 
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B.6 Conclusion 



In the past, definition of performance measurements of wireless telecommunications networks was focused mainly on 
satisfying the needs of the Equipment Vendor Performance Modelling and Development Engineering measurement user 
communities. The needs of the wireless telecommunications Network Operator were not always addressed. The 
Performance Measurement Definition process described in this annex addresses the needs of all measurements user 
communities. The process also provides additional benefits, including the following: 

• Allow measurement communities to specify their needs up front. 

• Allow measurement communities to prepare for and modify their measurement monitoring and reporting 
processes before product is released to them. 

• Allow measurement communities to know what actions they need to perform when assessing collected 
measurements. 

• Provides method for rejecting unrealistic goals and measurements. 

• Provides method for best architecting measurements into network devices. 

• Provides method for producing consistent measurement definitions. 

• Provides method for managing measurements life cycle including measurement creation, modification and 
obsoletion. 

The EGQM methodology may be used for: 

• analyse and assess performance areas that are not well understood or are highly complex; 

• non-straightforward cases where it is difficult to create useful measurement proposals; 

• an understanding of real value is required before useful measurement proposals can be created; 

• mine for missing measurements; 

• mine for conflicting, overlapping, or existing measurements that are no longer useful. 

In summary, the EGQM methodology may be used by any company to generate measurement definitions that can then 
be contributed to 3GPP SA5 for potential inclusion in the present document. 
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